Increasing effectiveness of surveys for digital health monitoring

ABSTRACT

In some implementations, a system dynamically adjusts an electronic form of an application based on user data and form data specified for the electronic form. Form data that specifies characteristics of the electronic form is initially received by a computing device. The form data specifies one or more rules configured to vary the content of the electronic form that is presented for different users. User data indicating characteristics of a user is then obtained by the computing device. An interactive element is then selected from among a set of multiple interactive elements based on the received user data and the rules in the received form data. A view of the electronic form is customized for the user by including, in the customized view, the interactive element. The view of the customized electronic form is then displayed by the computing device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/808,448, filed Mar. 4, 2020, now U.S. Pat. No. 11,244,104, which is acontinuation of U.S. application Ser. No. 15/935,276, filed Mar. 26,2018, now U.S. Pat. No. 10,621,280, which is a divisional of U.S.application Ser. No. 15/279,845, filed Sep. 29, 2016, now U.S. Pat. No.9,928,230, all of which are hereby incorporated by reference in theirentirety.

GOVERNMENT LICENSE RIGHTS

This invention was made with government support under grant numberHHSN261201500013C awarded by the National Institutes of Health. Thegovernment has certain rights in the invention.

FIELD

This specification generally describes technology related toapplications for computing devices.

BACKGROUND

Applications for computers, mobile devices, and other devices canprovide useful functionality to users. Users generally provide input onuser forms that are presented on user interfaces of applications. Userforms can include standard graphical user interface elements such ascheckboxes, radio buttons, or text fields to allow users to enter datathat is sent to a server for processing. For example, a text box canenable input of a single line of text (e.g., username, password), aradio button can enable a selection from a list of items, and a submitbutton enables an instruction for an application to perform a specifiedfunction. Frequently, user interfaces of application are staticallyconfigured to provide pre-defined user entry forms to a user.

SUMMARY

In some implementations, a system allows customized forms to bedynamically provided through an application. The system can include aninterface for an administrator to create a form, and then remotely makeadjustments to the form that take effect automatically and in real timeon user devices. The forms that an administrator creates can also bedynamically customized for each user. As a result, the same formdefinition can cause different interfaces to be shown to differentusers. Similarly, the same form definition can cause a different view tobe shown to the same user at different times, due to differences in thecontext of the user. These features and others discussed below giveadministrators the flexibility to provide multiple variations of a formthrough a single form definition and easily update the form for a largenumber of users. Similarly, the customizations for individual users canincrease user engagement with the application and reduce the burden onusers to enter data in the form.

Conventional applications often do not allow remote adjustment ofelectronic user forms or customization for individual users. As aresult, these forms have often failed to maintain consistent andmeaningful engagement with users. For instance, electronic forms ofapplications are often defined by client-side instructions that arepre-defined for a general population of users. In the case of healthcaresupport applications, electronic forms that are provided to a largepopulation of users often need to be adjusted for different users to beeffective. For example, forms may need to collect different types ofinformation at different precision levels in order to effectivelymonitor the individual physical and mental wellbeing across differenthealth conditions.

In some implementations, a system is capable of variably and dynamicallyadjusting electronic forms provided to users on an application in orderto improve collection of information related to different user-specificfactors. For example, different configurations of an electronic form canbe customized for users with different conditions or users withdifferent attributes. The use of customized electronic forms can thenallow for precise control over the type of data collected, techniquesused to collect data, and the content provided on the customizedelectronic form based on user attributes. As a result, engagementbetween users and the application are improved by individuallypersonalizing electronic forms provided on the application.

The server can customize the electronic forms by updating a formdefinition that configures the electronic form. The form definitionspecifies characteristics of the electronic form such as displayproperties, interactive elements to be included, and/or rules that varycontent of the electronic form for different users or user types. Theform definition can also specify different data types to be collected onthe user form (e.g., user input provided on the electronic form, dataobtained from third party APIs, data collected by devices wirelesslyconnected to the user device, or data collected by sensors of the userdevice). In addition, the form definition can also specify validationrules that specify types of valid or invalid input by the user.

The architecture of the system generally includes a user device that hasthe application installed, and an associated system where anadministrator can create and update a form definition for an electronicform that is provided to the user through the application. Theadministrator can design the electronic form using a configurationinterface that allows for customization of the electronic form byspecifying data types to collect using the electronic form and/or layoutparameters for displaying the electronic form on the application. Anapplication server can then process the administrator's design in orderto generate a form definition that includes rules for generating theelectronic form on the user device. The generated form definition isthen transmitted to the user device so that the user device candynamically generate the electronic form within the application withoutany intervention by the user.

The rules specified within the form definition can be used todynamically adjust the content provided on the electronic form. A clientdevice can interpret the form definition to adjust individual views ofthe electronic form, e.g., by changing the interface elements.that areincluded within an electronic form, or adjusting the precision level ofinformation provided on an electronic form. Adjustments can also be madeto multiple electronic forms such as adjusting the arrangement ofmultiple forms that are displayed on a single user interface.

Adjustments to electronic forms can be carried out in real-time withoutrequiring application updates (e.g., without altering the executablecode of the application). For example, the server may periodicallyprovide form definition updates to adjust an electronic form within theapplication without requiring a full application update. The applicationcan then process the updated form definition to adjust the electronicform while the user is using the application. In this regard, the systemcan automatically update electronic forms responsive to data received onthe user device.

If a user provides an invalid input into an electronic form, then theelectronic form can be updated to restrict submission of additionalinformation. In another example, if the application receives sensor datathat reflects user performance (e.g., pedometer data), then anelectronic form that requests the user to submit the number of steps maybe dynamically deactivated to reduce data redundancy.

In addition, the server can adjust the form definition based on datathat is received by the user device over a period of time. As anexample, a form definition adjustment can include rules that specifyupdates for specific user activities, user behaviors, and/or useractions on the user device, and how these actions vary over time. Afterthe server generates an initial form definition, the form definition canthen be adjusted based on monitoring the user's real-time activity onthe application. In this regard, adjustments can either be reactive torecent changes in a user's actions, or proactive in using patternrecognition analysis of historical user activity to anticipatesubsequent changes in user activity.

In one general aspect, this specification describes a method performedby one or more computing devices. The method includes receiving, by thecomputing device and over a computer network, form data that specifiescharacteristics of an electronic form, the form data including one ormore rules configured to vary the content of the electronic form that ispresented for different users; obtaining, by the computing device, userdata indicating characteristics or activities of a user of the computingdevice; selecting, by the computing device, an interactive element fromamong a set of multiple interactive elements based on the user data andthe rules in the received form data; customizing, by the computingdevice, a view of the electronic form for the user of the computingdevice by including, in the customized view, the interactive elementselected based on the user data and the rules in the received form data;and displaying, by the computing device, the view of the electronic formthat is customized for the user of the computing device.

One or more implementations can include the following optional features.For example, in some implementations, the form data is configured tocause the computing device obtain (i) user input to the electronic form,and (ii) additional data from one or more additional data sources, theadditional data including passively-sensed sensor data, actively-sensedsensor data, data from a sensor of the computing device, data from adevice wirelessly connected to the computing device, or data from athird-party API.

In some implementations, the one or more rules include a first rule, andwhere the first rule specifies one or more triggers for application ofthe first rule, one or more conditions to evaluate for the first rule,and one or more actions to be performed based on a satisfaction of atleast one of the one or more conditions.

In some implementations, the electronic form includes multiple sectionsdisplayed in a sequence, where the one or more rules customize thecontent of the sections or the sequence of the sections based on userinput provided to the electronic form.

In some implementations, the form data defines at least one validationrule that specifies requirements for user input to a portion of theelectronic form; where the method further includes: receiving an a userinput through the electronic form; determining that the user input doesnot satisfy the at least one validation rule; and restricting submissionof data through the electronic form based on determining that the userinput does not satisfy the at least one validation rule.

In some implementations, the form data specifies a data type for userinput collected through the electronic form; where the method furtherincludes: selecting, by the computing system, an interactive element toreceive user input of the specified data type, the interactive elementbeing selected from among multiple different interactive elements thateach receive input of the specified data type; and including, by thecomputing system, the selected interactive element in the customizedview of the electronic form.

In some implementations, selecting the interactive element includesselecting the interactive element from among multiple differentinteractive elements that each receive user input of the specified datatype at a different level of precision.

In some implementations, the form data is provided to an applicationthat runs on the computing device, and the view of the electronic formis provided by the application; where the method further includes:receiving an updated form data over a network after displaying the viewof the electronic form that is customized for the user of the computingdevice, the updated form data indicating a change to the electronicform; and displaying an updated view of the electronic form through theapplication in response to receiving the updated form data beforerestarting the application or updating executable code of the program.

In some implementations, customizing the view of the electronic formincludes: determining a level of difficulty required for a user tocomplete data entry to the electronic form; determining that the levelof difficulty exceeds a threshold level of difficulty; and in responseto determining that the level of difficulty exceeds the threshold levelof difficulty, adjusting the content in the customized view of the formto reduce the level of difficulty below the threshold level ofdifficulty.

In some implementations, customizing the view of the electronic formincludes: determining that the form data instructs collection of data ofa particular data type; obtaining, from the computing device, the dataof the particular type based at least in part on sensor data from thecomputing device; based on obtaining the data of the particular type,customizing the view of the electronic form to omit a region forrequesting data of the particular type from the user.

In another general aspect, this specification describes a differentmethod is performed by one or more computers. The method includes:providing, by the one or more computers, a user interface for designingan electronic form for collecting data about a user; receiving, by theone or more computers, data indicating user input through the userinterface, the user input indicating one or more data types for data tocollect using the electronic form and one or more layout parameters forthe electronic form; generating, by the one or more computers, form datathat specifies one or more rules for customizing the electronic form;and providing, by the one or more computers, the form data to acomputing device, the form data being configured to cause differentversions of the electronic form to be presented to different users.

In some implementations, the method further includes: receiving, fromdevices of multiple users, data collected by the electronic formassociated with the form data, the received data including informationof a particular data type that was collected in different data formatsfrom different users; converting the received information of theparticular data type to a common data format; and storing the converteddata.

In some implementations, converting the received information includes:accessing data indicating a hierarchy of categories; identifyingdifferent categories in the hierarchy that correspond to the differentdata formats; and converting the information of the particular data typeusing relationships of the categories in the hierarchy that correspondto the different data formats.

In some implementations, providing the form data to the computing deviceincludes: providing the form data to the computing device such that theform data updates a form definition in an application installed at thecomputing device without any action by the user of the computing device.

In some implementations, the form data includes one or more state rulesfor the electronic form, where the one or more state rules cause thecomputing device to select, from among multiple assessment algorithms, aparticular assessment algorithm according to data collected using theelectronic form.

In some implementations, providing the user interface includes providinga user interface that indicates a set of pre-defined database fields;where receiving the data indicating the user input includes receivingdata indicating selection of a particular database field; and wheregenerating the form data includes including, in the form data, anelement of the form that is personalized for a viewer of the electronicform using data for the viewer from the particular database field.

In some implementations, providing the user interface includes providinga user interface that enables a user to (i) update the form data tochange characteristics of the electronic form and (ii) publish theupdated form data to cause an application installed on remote computingdevices to update the characteristics of the electronic form in realtime.

In some implementations, the electronic form includes multiple sectionsshown in different views; where receiving data indicating the user inputincludes receiving data indicating rules configured to cause differenttransitions among the sections of the electronic form based on datacollected by the electronic form.

In some implementations, generating the form data includes generatingthe form data to direct a computing device presenting the electronicform to capture data from one or more additional data sources thatincludes at least a device wirelessly connected to the computing device,a third-party API, and a sensor of the computing device.

In some implementations, providing the user interface includes providinga user interface that permits a user to specify one or more validationrequirements for data entered into the electronic form.

Other embodiments of these aspects include corresponding systems,apparatus, and computer programs, configured to perform the actions ofthe methods, encoded on computer storage devices. A system of one ormore computers can be so configured by virtue of software, firmware,hardware, or a combination of them installed on the system that inoperation cause the system to perform the actions. One or more computerprograms can be so configured by virtue having instructions that, whenexecuted by data processing apparatus, cause the apparatus to performthe actions.

The techniques described herein can provide a number of advantages andbenefits. For example, a single form definition can be provided tomultiple user devices and include embedded rules. The rules maycustomize the display of an electronic form each time it is displayed.Similarly, the rules may allow an application to customize the form foreach user and each device, so the same form definition providessignificantly different presentations depending on the attributes of theuser, the capabilities of the device, and/or different user inputs tothe form. By maintaining and distributing form data in this manner, thestorage requirements and complexity of distribution can be significantlyreduced. In addition, the system can permit transmit updated formdefinitions to cause the forms displayed on user devices to be updatedremotely and in real time or near real time. Additional advantagesresult from the ability to eliminate unnecessary interactions that arenot likely to assist the user. By eliminating these unnecessary outputs,the user devices conserves battery power, processing cycles, and networkbandwidth.

The details of one or more implementations are set forth in theaccompanying drawings and the description below. Other potentialfeatures and advantages will become apparent from the description, thedrawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram that illustrates an example of a process fordynamically generating an electronic form based on a form definition.

FIG. 2 is a diagram that illustrates an example of a system that iscapable of generating dynamic electronic forms.

FIG. 3 is a diagram that illustrates an example of a configurationinterface for customizing form definitions.

FIG. 4 is a diagram that illustrates an example of a configurationinterface for customizing precision levels for data to be collected onan electronic form.

FIG. 5 is a diagram that illustrates an example of a process forprocessing user input on an electronic form in relation to a set ofhierarchal data attributes.

FIG. 6 is a diagram that illustrates an example of a performanceanalysis report for an electronic form.

FIG. 7A is a diagram that illustrates an example of a process fordisplaying a customized electronic form to a user.

FIG. 7B is a diagram that illustrates an example of a process forgenerating a form definition for customizing an electronic form.

FIG. 8 is a block diagram of computing devices on which the processesdescribed herein, or portions thereof, may be implemented.

In the drawings, like reference numbers represent corresponding partsthroughout.

DETAILED DESCRIPTION

In general, a system allows customized forms to be dynamically providedthrough an application. The system can include an interface for anadministrator to create a form, distribute it to various user devices,and then remotely make adjustments to the form that take effectautomatically and in real time on the user devices. The forms that anadministrator creates can provide a dynamically customized view for eachuser. As a result, the same form definition can cause differentinterfaces to be shown to different users. Similarly, the same formdefinition can cause a different view to be shown to the same user atdifferent times, due to differences in the context of the user. Thesefeatures and others discussed below give administrators the flexibilityto provide multiple variations of a form through a single formdefinition and easily update the form for a large number of users.Similarly, the customizations for individual users can increase userengagement with the application and reduce the burden on users to enterdata in the form.

As described throughout, form definitions can be used to configure andadjust an electronic form provided by the application. The formdefinition is initially specified by a system administrator on aconfiguration interface that allows the administrator to customize, forexample, data types to collect using the electronic form and/or layoutparameters for the electronic form. The specified form definition isthen transmitted to the user device to either dynamically generate oradjust the electronic form of the application without any interventionby the user.

The architecture of the system provides various improvements indynamically configuring an application to provide customized electronicforms to individual users, which often requires significant computingand storage resources on client devices. For example, in order togenerate a new electronic form to be displayed on an application, asoftware update for the entire application is usually necessary becauseform data is often pre-configured within application code. Thearchitecture of the present system, however, utilizes configurable formdefinitions that process a set of rules that dynamically configures anelectronic form based on data collected on the application. Forinstance, the client device may process form definition data to generatemultiple customized electronic forms without requiring anapplication-wide update from the server. As described below, thisenables the application to utilize less storage space on the clientdevice and less network bandwidth since the entire application does notrequire an update each time a new version of an electronic form is to begenerated.

In addition, the architecture of the system enables the system toaddress various problems that arise in the networked environment ofclient-based applications. As an example, application configurationsprovided by servers are often static for significant periods of time andmay require major updates to application code. This can be timeconsuming for application developers and end-users since it requiresperiodic manual reconfiguration of applications, and periodic deliveryof the reconfigured applications. As described in more detail below, thearchitecture of the system addresses this problem by, among othertechniques, employing the use of form definition data that uses acollection rules to automatically adjust electronic forms to be providedon an application without requiring changes to the entire applicationconfiguration. This enables an administrator or a developer to adjustthe information collected by the application with limited intervention.Moreover, processing of these rules by the system can automatically anddynamically alter a user's experience with the application withoutrequiring users to manually download application updates.

Further, the implementations discussed below can provide ongoingcustomization of user experiences in an efficient manner, reducingcomputation needs, power consumption, and other resource requirements.Processing rules within form definition data for a large set of userscan be done efficiently by limiting processing of rules for a given userto a particular subset of rules applicable to the user. Even for rulesin the subset, processing can be limited by using the trigger as athreshold test, so that further processing of the rule is not performeduntil and unless the trigger is detected.

Another problem that often arises in networked environments is thatapplications are often unable to provide effective real-time adjustmentsthat are appropriate for a user's particular circumstances. For example,while some applications can process historical data and provide analysesand recommendations, such applications are often unable to dynamicallyadjust (i) the content that is provided to the user on the application,and (ii) the arrangement specifications of the application (e.g., typesand characteristics of interface elements, color schemes used in stylespecifications, etc.) without requiring a manual application update. Asdescribed in more detail below, the architecture of the system canaddress this problem with, among other techniques, the use of acollection of rules that associate triggers and conditions with actionsto be executed by the system. The scope of each rule can be manipulatedbased on selections of the triggers, conditions, and system actionsspecified by an administrator. In this regard, some rules can be appliedto make general adjustments (e.g., providing notification badges, oradjusting content provided), whereas other rules can be applied ascontext-specific changes that respond to user behavior.

Yet another problem that often arises in networked environments is thatthird-party organizations that create content to be provided on anapplication developed by a separate entity often have limited ability tocontrol as to how a user interacts with their content. For example,third-party organizations often have to adjust content creation tosatisfy application requirements set forth by application developersthat provide updates to an application. As described in more detailbelow, the architecture of the system can address this problem, byproviding a configuration interface to an administrator associated witha third-party organization that enables the customization of forms to beprovided in third-party application modules accessible on theapplication by a user. The configuration interface enables theadministrators to make adjustments using a user-friendly interfacerather than having to adjust the application code directly. In addition,because the system architecture enables the creation of multiplethird-party application modules to be provided within the sameapplication, different third-party organizations can configure andadjust the content to be displayed within their respective applicationmodules.

FIG. 1 is a diagram that illustrates an example of a process 100 fordynamically generating an electronic form based on a form definition.The form generation techniques described in this document can be usedfor any appropriate application. For clarity in illustration, examplesof forms related to healthcare management are provided and discussed,but the technology is not limited to those uses or data types.

In the example depicted in FIG. 1, a health application on a user deviceis used to periodically monitor joint pain experienced by a user of theuser device in relation to activity level of the user. The healthapplication accesses a form definition 112 that describescharacteristics of an electronic form to collect data about the user.The form definition 112 includes information that allows the applicationto provide user interfaces that request certain data from the user. Theform definition 112 can also cause the application to obtain otherinformation about the user, for example, actively-sensed sensor data,passively-sensed sensor data, information through third-partyapplication programming interfaces (APIs), and so on. Based on the formdefinition 112, the application in the example of FIG. 1 provides usersurveys for the user to provide input about pain levels experienced bythe user. The user surveys include different views or user interfacesthat accept user input. Input received on the electronic forms can thenbe collected, analyzed, sent to a server system, and used by theapplication to assist the user.

The user is initially directed to a start a user survey on an interface110 a. Upon receiving a user input to initiate the survey, the systemdynamically generates a user interface 110 b that includes a formelement 114 to enable the user to rate a joint pain level. The formelement 114 is generated based on a configuration specified by the formdefinition 112. For example, the form definition 112 may specify a typeof data to be gathered, such as joint pain level, and may also specifylayout parameters about how the user interface 110 b should appear. Inaddition, the application can customize one or more elements of the userinterface 110 b for the specific user. In some instances, the formdefinition 112 itself describes different variations of an interfacethat can be provided through rules embedded in the form definition 112.In addition, or as an alternative, the application that generates a viewof the form may apply customizations that are not specifically indicatedby the form definition 112.

In the user interface 110 b, the application determines which of severalform elements 114 a, 114 b, or 114 c to display based on a set of rulesgoverning the configuration of the electronic form. Each of the formelements 114 a, 114 b, or 114 c can be used to obtain data about a jointpain level. However, the different form elements 114 a, 114 b, or 114 cobtain the information at a different level of precision, and differentelements may be appropriate for different users. The form element 114 arepresents an example of a low-precision element that only acceptsbinary input from the user (e.g., “Yes” for experiencing joint paint or“No” for not experiencing joint paint). The form element 114 brepresents a medium-precision element with a slider element that allowsa user to specify a value within a finite spectrum of values (e.g., avalue along a range as illustrated in FIG. 1). The form element 114 crepresents a high-precision element that allows the user to provide anumerical value within a specified integer range (e.g., 0 to 100).

The application may select one of the electronic form elements 114 a-114c to display on the interface 110 b based on rules specified by the formdefinition 112. The form definition 112 may specify various rules thatspecify a selection of a particular visual format based on usercharacteristics, user history, requirements for a health program thatthe user participates in (e.g., program objectives, milestones, etc.),and a total burden imposed on the user, and other factors. As anexample, a user profile may indicate that the user has previously haddifficulty answering questions on electronic surveys, or that the userhad a low rate of completing surveys. The form definition 112 mayspecify that for users with low completion rates, the form element 114 ashould be selected since this form has the lowest complexity with abinary selection. In another example, the user history may indicate thata user's joint pain level, indicated by the previous user inputs, hasvaried greatly. The form definition may specify that if the variation inprior inputs exceeds a certain level, then the form element 114 c shouldbe selected to provide the user with greater precision in providing apain assessment. In other examples, the selection may also be based onuser preferences, the type of medical condition monitored for the user,among other factors.

When customizing a user interface for a user, the application may applyany of a variety of rules. Some rules may be general or default rules,such as an application-level rule that attempts to maintain a consistentvisual style or to keep the complexity of a form below a maximum level.Other rules may be specific to individual forms, as indicated inspecific form definitions. The form-level rules may be used in additionto the general rules, and may override the general rules for thepurposes of presenting specific forms.

A variety of customizations can be made to personalize a view of anelectronic form for a particular user. For example, the application canalso determine whether or not to request certain information from auser. Based on the data available, or information about the currentcontext of the user, the application may omit certain form elements fromthe user survey. This determination can be based on different types ofdata that are received by the application. For example, an applicationcan obtain data from sources other than user input, such as sensor dataor stored data from the user's device. This additional data can be usedto meet the data collection requirements indicated by the formdefinition 112, and can also adjust the view of the form.

In FIG. 1, the application receives pedometer data 116 indicating theuser's current number of steps taken and location data 118 indicatingthe user's current location. The form definition 112 may direct theapplication to provide an interface to collect activity data andnutrition data from the user. However, the application can determinethat the pedometer data 116 provides the activity level informationneeded, and so the user does not need to enter input. Because thepedometer data 116 already indicates the activity level of the user, theapplication no longer requires user input to identify a measuredactivity level. The application generates the user interface 110 c toexclude the “activity level” form element, thus simplifying the viewshown to the user. In this this regard, data received from externalsources (e.g., wearable devices wirelessly connected to the user device,third party application programming interfaces (APIs), and sensors ofthe user device) can be used to dynamically generate electronic forms tominimize unnecessary user input. If the pedometer data 116 or other datawere not available to the application, then a different user interfaceincluding a request for activity level data would be provided. The formdefinition 112, may indicate that the application should acquireinformation from certain data sources. In addition, or as analternative, the form definition 112 may indicate certain types of datato acquire, and the application may include data separate from the formdefinition 112 that specifies sources from which to acquire that data,such as a mapping between data types and data sources.

As shown in FIG. 1, a single form definition 112 can define thecharacteristics of multiple user interfaces 110 a-110 c. Together, theseuser interfaces 110 a-110 c can provide a multi-stage form with multipleviews presented sequentially. According to the rules in the formdefinition 112, the content of each individual view, the sequence of theviews, and whether certain views will be shown to a user can vary fromone user to the next. Similarly, the rules in the same form definition112 may cause differing views to be provided to the same user atdifferent times.

As described throughout, rules embedded within the form definition canbe used to customize an electronic form for a particular user. Each rulecan be associated with one or more triggers, one or more evaluationconditions, and one or more specified actions to be automaticallyperformed by the system. Satisfaction of the triggers of a rule can beused to automatically initiate an evaluation of associated conditions ofthat rule. The system then performs the corresponding system actions inresponse to determining that at least one of the conditions has beensatisfied. The embedded rules can thus be used by an application todetermine, for example, which of multiple forms should be provided, whena form should be displayed, the content and appearance of a form, whichdata should be requested through a form, what level of precision orcomplexity should be used in the form, and other form characteristics.Some embedded rules may be used generally to select or schedule a formfor display, while other rules may be used specifically to customize aform for an individual user. This technique can be used selectivelyperform system actions related to the electronic form based on thetriggers and conditions specified by individual rules within arepository of rules.

The repository of rules may include different types of rules that adjustform generation in various ways. For instance, the individual ruleswithin the repository of rules can vary in scope and in hierarchy toenable an application to adjustably generate an electronic form. As anexample, certain rules can be “state rules,” which enable the system toselect an appropriate assessment algorithm from among multipleassessment algorithms based on the data collected using an electronicform. In this example, each state rule may specify the use of adifferent assessment algorithm as a system action based on thesatisfactions of different triggers and/or conditions. In anotherexample, other rules can be “validation rules,” which enable the systemto determine whether a user input on the electronic form is valid basedon the data type associated with the conditions specified by eachvalidation rule. While state rules and validation rules are two examplesof different types of rules, other examples are also possible.

State rules can be defined in any of various ways. As an example, astate rule can have one or more triggers organized in an unordered list.If any of the specified triggers occurs, the state rule is executed.Similarly, a state rule can have one or more conditions organized in anordered list. The conditions can be evaluated in sequence, until the oneof the condition expressions evaluates to “true,” indicating that thecondition is satisfied. The application can then cause the actionassociated with the satisfied condition to be performed. Each conditionmay have a single condition expression. In some instances, a conditioncan have a “null” value or “true” value indicating that the condition isalways satisfied and always evaluates to true. For each condition, oneor more actions are defined. These actions can be organized in anordered list or be prioritized in another manner. When a condition isselected as being satisfied (e.g., is the first in an ordered list thatevaluates to true), each of the actions will be performed, in thesequence defined by ordered list of actions.

Other variations of state rules can be used. For example, in someimplementations, certain actions may have additional conditions thatmust additionally be satisfied before the action is performed. Asanother example, some state rules may be defined so that each conditionis evaluated after a trigger is detected, and the actions specified foreach condition that is satisfied can be performed. In some instances,actions may set, clear, or adjust values for parameters of a form beingdisplayed. The actions may change various display configurationproperties. The various values or properties can be specified by a type,an identifier, and/or a path through the hierarchy of elements specifiedin the form data.

As described throughout, a customized electronic form can be used fordifferent types of health management applications. For example, in someimplementations, form customization enables the system to provide riskassessment forms that provide directed questions to obtain informationabout a user's present health condition. The information obtainedthrough the customized electronic form can then be used to evaluatehealth risks associated with a known medical condition of the user. Thetypes of customized forms can typically be used to improve a user'sprogress through a particular health module. In another example, formcustomization enables the system to obtain user feedback data on theeffectiveness of the actual health module. For instance, electronicforms can be provided to request a user's feedback on a treatmentprograms specified by a health module they are presently participatingin. The two examples above illustrate that form customization can beused to both gain information relating to a user's performance within ahealth program, as well as the effectiveness of the health program. Theobtained information can be enable an administrator to identify changesto either a user's individual treatment plan (e.g., an adjustment ofindividual user's goals or actions), or changes to the way in which ahealth program is provided to a collection of users (e.g., adjusting acare plan for users that are diagnosed with the same medical condition).

FIG. 2 is a diagram that illustrates an example of a system 200 that iscapable of generating dynamic electronic forms. The system 200 generallyincludes an administrator system 210, an application server 220, and auser device 230 connected over a network 205. The administrator system210 provides a configuration interface 212 that allows an administratorto design electronic forms that are provided for output on anapplication 232 running on the user device 230. As described above, thesystem 200 enables data exchange between the administrator system 210,the application server 220, and the user device 230 such that electronicforms displayed on the applications 232 can be dynamically constructedand adjusted based on forms designed by the administrator using theconfiguration interface 212.

In general, the application 232 can support multiple program modulesprovided by different third-party organizations separate from theorganization associated with the application server 220. The variousprogram modules can include health modules. A health module canrepresent any type of program to measure, maintain, or enhance aperson's physical and/or mental wellbeing. Health programs can be usedfor monitoring, treatment, research, coaching, teaching, and otherpurposes. Each program can provide a user experience that benefits theuser through the use of the application. A user can enroll in multiplehealth modules within the application 232 that are provided by differentorganizations. In this regard, the system 200 enables administratorsfrom different organizations to design customized electronic forms thatare provided within different programs of the same application.

In step (1), the administrator system 210 initially receives form designdata from the application server 220. As noted above, the administratorsystem 210 can be a client device of a third-party administrator thatcreates or maintains a program provided through the application server220. The design data may include pre-generated form design templatesthat specify text, required text, and/or associated rules. The designtemplates may be specified by a particular medical condition, aparticular health program in which a group of users have participatedin, or an entity or organization that provides healthcare services to agroup of patients. For example, design templates for an employer mayinclude form data that relates to specific health programs orinitiatives supported by the employer. In another example, designtemplates for a particular medical condition may include form data forcollecting patient information in relation to treatment of theparticular medical condition.

The administrator may then use the configuration interface 212 to designcustomized electronic forms to be provided for output on the application232. The customized electronic forms may include a customized selectionof interface elements, content to be provided to the users, anddifferent data types to be collected on the electronic forms. Morespecific descriptions related to the configuration application areprovided below with respect to FIG. 3.

The administrator to configure many permutations of form experienceswith a single form definition. This allows for a high degree ofpersonalization and engagement tailored for individual users. Dependingon the choices that a particular user makes when viewing the form, andthe data that is entered in the form, the appearance and behavior maychange. For example, the state rules discussed above can definedifferent combinations of content and input fields to display, dependingon the user's responses and other user information obtained from othersources besides the input fields of the form. In some implementations,an administrator may define a set of rules that provide acondition-based survey with branching logic or cascading user inputfields, where the entry in first field governs the presentation of asecond field in the form.

In step (2), the customized electronic forms designed by theadministrator are then transmitted to the application server 220. Thecustomized electronic forms may include designed user interfaces thatinclude the customized electronic forms, rules that indicate thebehavior of forms or variations of the forms for different users orscenarios, or various types of instructions that cause the application232 to display the customized forms on the application 232. The datareceived from the administrator system 210 may then be aggregated andpackaged by the application server 220 based on a set of storedapplication configuration data 222.

In step (3), the packaged data can then be transmitted from theapplication server 220 to the user device 230 as form data thatconfigures the application 232 to provide the customized electronicforms designed by the administrator for display to the user. The updateddefinitions in the form data are provided automatically and remotely tothe user devices, substantially in real time. In some implementations,when the form data for a program module is updated, the packaged datacan be pushed to each user device associated with a user who hassubscribed to that program module. In some implementations, theapplication on the user device periodically requests updates to formdata for the programs that are subscribed.

The dissemination of form data to the user devices requires no action onthe part of the subscriber, e.g., there is no need for applicationupdates, re-login to an application, or even restarting of theapplication. Once form data is received at a user device, a new orupdated form can be provided as soon as display of the form is triggeredaccording to the rules in the form data.

This ability to almost instantly provide form updates also assistsadministrators to preview and test form configurations. Theadministrator may cause an updated form definition to first be providedonly to certain devices designated for testing, or to be active on onlycertain devices. This limited distribution phase allows an administratorto see the results of updated forms, on the actual types of deviceswhere the forms will be seen, before disseminating the updated form tosubscribers. With these techniques, administrators can conduct rapidprototyping of forms for user experience and usability testing. Forexample, form updates can be rolled out to small groups, assessed,updated, and then provided to a broader set of users.

The form data can include form definitions that allow user devices thatreceive the form data to customize views of the forms for specificusers, as described above, and rules that specify triggers andconditions in which the customized forms are provided for display to theuser.

In some instances, the form data may also include instructions thatcause the user device 230 to obtain information through the user inputof the customized electronic forms, through other devices that arewireless connected to the user device 230 (e.g., wearable devices),third party application programming interfaces (APIs), or sensors of theuser device 230. For health-related forms, this allows a single form tocollect objective, un-biased information about a user in addition toself-reported data obtained as user input to the form. In otherinstances, the form data may include specifications that configure oradjust the content of the customized forms, or the arrangement ofcustomized forms on the interface of the application 232. For example,the form data may specify multiple sections for a particular electronicform, and rules included within the form data may customize the contentof each section or the sequence of the sections based on user inputprovided on the electronic form.

In some implementations, the sections of a form can be nested. Forexample, the form data may specify different views or portions of aform, and groupings or other relationships of the different views. Forexample, form data may specify that different sections of a form shouldbe shown, depending on a user's responses to a first section of a form.For example, an initial view of a form may ask a user to inputinformation about exercise. The form data may specify that if the userenters a type of exercise that is typical for that user, then the formshould then proceed to display the next section, e.g., asking aboutnutrition. The form data may specify that if the user enters a type ofexercise that is new for the user, one or more nested sections relatedto exercise should be provided, e.g., congratulating the user for tryingsomething new, asking about how the user liked the new activity, and soon. The form data may organize form sections into groups or hierarchiesthat specify different nested elements that are selectively displayedaccording to the current information about the user. Different views canbe displayed in sequence as indicated by the form data.

In other examples, the form data may define validation rules thatspecify requirements for user input to portions of the customizedelectronic forms. For instance, the form data may specify a specificdata type for a particular electronic form (e.g., numerical input, orinput in a particular numerical range). The form data may includeinstructions to adjust the configuration of the customized form inresponse to determining that a user's input is invalid or does notsatisfy the requirements set forth by the validation rules. As anexample, an electronic form for physical pain level measurement mayrequest that a user may provide a description of the pain experienced bythe user. In this example, a validation rule may indicate common termsthat are related to physical pain measurement, which would representacceptable input in a free-form text input area. Inputs that do notsatisfy the criteria in the validation rules, such as a user inputrelating to depression (e.g., emotional pain), can be identified asinappropriate. In this instance, the user may have misunderstood thequestion presented on the initial form. In response to detecting theinappropriate input, the application 232 may prompt the user to correctthe answer, or may prevent the user from continuing or submitting theform.

In step (4), data collected by the application 232 through forms isprovided to the application server 220. The collected data 224 caninclude user input (e.g., data indicating user responses or interactionswith a view of the form) as well as other information that a formdefinition directs the application 232 to obtain. The application 232may periodically collect user input data provided to the customizedelectronic forms (e.g., text submitted, user selections from a list ofpresented options, etc.) and other input data that is associated withthe user's participation in the electronic form (e.g., total time takento complete the electronic form, other background applications used bythe user while completing the electronic form, etc.). The application232 may also collect other data separate from user input, as directed bythe form definition. For example, the application 232 may collectpassively-sensed sensor data or actively-sensed sensor data. The datacan be obtained from additional data sources such as the sensors of theuser device 230, a wearable device wirelessly connected to the userdevice 230 (e.g., through Bluetooth), or from a third-party API. Thecollected data for each electronic form is then transmitted to theapplication server 220 for storage and analysis.

In step (5), the application server 220 then generates performancereports for the customized electronic forms based on the user input datacollected on the application 232. The performance reports may includesummary information indicating user patterns while completing thecustomized electronic forms (e.g., amount of time spent on thecustomized electronic forms), or analytical determinations based on theuser patterns (e.g., effectiveness of the customized electronic forms).More particular descriptions related to the performance reports that aregenerated for the customized electronic forms are provided below withrespect to FIG. 6.

The performance reports generated by the application server 220 are thentransmitted to the administrator system 210 for analysis. In someinstances, the performance reports may be used to adjust formdefinitions for customized electronic forms in order to improve theengagement of the user on the customized electronic forms. Adjustmentsto the electronic forms may be made manually by the administrator on theconfiguration interface 212, or automatically by either theadministrator system 210 or the application server 220 based on theinformation indicated by the performance reports. In the first example,the administrator can adjust the form definition by providing input onthe configuration interface 212 (e.g., adding or removing interfaceelements of an existing form definition, adjusting the content to bedisplayed, etc.). In the second example, automatic adjustments to theform definitions can be processed based on the identification ofspecific types of user input data (e.g., not receiving any user input ona particular electronic form for a long period of time, indicating thatit may no longer be relevant to the user). In addition, as describedabove, adjustments can be processed in real-time such that an existingelectronic form may be adjusted as the user is providing user input onthe electronic form.

In some implementations, the administrator can use the configurationinterface 212 to configure an entire health risk assessment (HRA) forthe user. The HRA can evaluate health assessments and recommendationsbased on data collected through the application 232 (e.g., passively oractively sensed data, data obtained from third-party APIs, etc.) inaddition to self-reported user input data on the electronic form.Because the configuration interface 212 enables the creation ofdifferent types of electronic forms, the administrator may configuredifferent HRA forms for a single health module that is run within theapplication 232. A health module can represent any type of periodicmonitoring or treatment program that the user can participate in throughthe use of the application. For example, a user can enroll in multiplehealth modules within the application 232. In addition, theadministrator can configure form definitions to specify rules thatdirect the selection of different HRAs based on the data collected onthe application 232.

In other implementations, other types of clinical assessments can beincorporated into the content of the electronic forms. For example, theadministrator may configure clinical outcome assessments (COA), patientreported outcomes (PRO), among other types. As described above, rulesspecified within the form definition can be utilized to provide specifictypes of output in response to receiving different types of user data.In this regard, the administrator can customize the electronic form tothe specific needs of a particular healthcare provider or entity inorder to perform a large variety of patient monitoring functions.

In some implementations, the administrator can also use theconfiguration interface 212 to configure an entire diagnostic tool (EDT)with the use of customized electronic forms. For instance, theadministrator may create a diagnostic form with a form definition thatspecifies rules for predicting a diagnosis for a user based on inputprovided on the user. As described above, the particular diagnosticevaluation applied can be varied based on the different types of datacollected on the diagnostic form.

In other implementations, the administrator can configure automatedclinical decision support features into the electronic forms using theform definitions configured on the configuration interface 212. Forexample, the configured form definition can include rules thatautomatically provide clinical recommendations for display on theelectronic form based on the contents of the data collected by the user.The rules can associate different types of input data that areassociated with particular medical conditions with the correspondingclinical recommendations that are often provided as solutions to typesof symptoms experienced by the user. As an example, if a user submits onthe electronic form that he/she is experiencing pain, a rule can beconfigured to automatically provide a recommendation to take a painrelieving medication in response to receiving the user input.

FIG. 3 is a diagram that illustrates an example of a configurationinterface 310 for customizing form definitions for electronic forms. Theinterface 310 may be presented to the administrator on the configurationinterface 212 described above. The interface 310 allows an administratorto have fine-tuned control over various aspects of an electronic form.For instance, in region 312, the administrator may select a data typefor the user input to be received on the electronic form (e.g., textinput, voice input, etc.), and a data source for the user input to bereceived on the electronic from (e.g., user input data received throughthe keyboard of the user device 230, input data received from otherdevices wirelessly connected to the user device 230, input data receivedfrom third party APIs, input data received from sensors of the userdevice 230). In some instances, the administrator may also specifywhether the input data to be collected is passively sensed data,actively sensed data, or a combination of both.

In region 314, the administrator may select from among a set ofpre-generated templates for the electronic form. The pre-generatedtemplates may be obtained from prior configurations of other customizedelectronic forms that are stored locally on the administrator system210, or other configurations that are stored remotely on the applicationserver 220, e.g., configurations made by other users. As an example, theadministrator may select a form template from a repository of formtemplates that categorizes the templates across various categories(e.g., health program, affiliated entities/organizations, associatedmedical conditions, etc.). After importing a form template, theadministrator may also make additional updates to customize theelectronic form that is being designed. For example, an administratormay begin with a standard template to quickly obtain a functional formfor acquiring data of a needed type. The administrator may thencustomize the appearance and behavior of the form, e.g., selecting thecolor scheme, logos, and other information. In this manner, theadministrator can quickly design an entire form, with appropriatecustomized branding to set it apart from others available through theapplication 232, without writing any computer code.

In region 316, the administrator may select the user interface elementsto be included within the electronic form to be designed. For example,the administrator may include images, buttons, text boxes, lists, andradio buttons, among other types of common interface elements. Theadministrator may also specify ways in which the user interacts witheach interface element. For example, the administrator may configurerules that adjust the availability of certain interface elements basedon the satisfaction of specified triggers or conditions associated withthe interface element. As an example, a particular interface element mayonly be available once the user has provided a valid user input to apreceding user interface element within the electronic form.

In region 318, the administrator may adjust the arrangement of interfaceelements and/or the interaction rules of interface elements within theelectronic template through a drag and drop visual layout. For example,as depicted in FIG. 3, the administrator can configure locations in theelectronic form where the selected user interface elements should beplaced, specify field values for certain elements, or provideinteractive elements that helps the user transition between differentparts of the same electronic form, or different electronic forms. Insome instances, the administrator may also specify embedded conditionallogic into interface elements for performed pre-determined calculationsbased on the user input provided on other interface elements (e.g.,calculating a body mass index based on the mass and height submitted bythe user on two preceding fields).

In region 320, the administrator may adjust the presentation of theelectronic form. For example, the administrator may adjust the size ofinterface elements, colors of the interface elements, fonts of text tobe displayed, and/or other graphical attributes associated with theinterface elements (e.g., shadows, line thickness, padding, margins,orientation, etc.). An administrator may also adjust the timingassociated with when the electronic forms are provided on theapplication. For example, the administrator may adjust time delaysbetween a designated user input that initiates a display of anelectronic form, specify triggered events that either cause anelectronic form to be displayed or removed from display, among othertypes of settings.

The configuration interface 310 may additionally include a progressionor preview tab that enables the administrator to view progressivechanges made to the draft electronic form (e.g., history of changes madefrom a pre-existing template) and/or a preview of electronic form withinthe user interface of the application 232.

The administrator may specify various user-specific customizations thatshould occur. For example, the configuration interface 310 may provide alist of database fields from which data may be obtained at the time ofform presentation. These fields may represent characteristics that canbe obtained from a user profile, e.g., a field [Subscriber_Name] or[SubscriberProfileImage]. When a view of the form is generated, theapplication 232 obtains the current information from the user's profile,or other data source referenced, to generate a custom view of the form.The database fields may also provide information that helps motivate andengage a user. For example, the form could include a sentence such as“[Supporter] thinks it's important for your cancer treatment to completethis survey,” where the [Supporter] field is replaced by a name offriend of note from that person could also be obtained and presented inthe view of the form.

The present techniques can facilitate efficiently and effectivelyconducting large-scale data collection projects through surveys. Othersystems that use hard-coded forms often lack the flexibility to controlto edit the survey or add new survey questions after deployment. Withthe techniques disclosed herein, a versioning technique can be used toallow multiple versions of a form. When the administrator edits anexisting data collection form, validation rules automatically determinewhether there are a subscriber has utilized this form to submit data. Ifnot, then the form is edited and saved as the same form version in thedatabase. If a user has submitted the form previously, then a newversion of the form is automatically saved. Any subscriber presentedwith the form will automatically be presented with the newer version ofthe form. However, if an existing subscriber edits or views a form thatthey saved using the older version, the system correctly presents theolder version of the form. This allows seamless updates for users thathave not yet used a form, while preserving the entered data and progressof users who have invested time in filling out the form already. Thissolves a business problem for large survey projects, which frequentlyneed updates and additions. As a result, a correction to a spellingerror, rephrasing a question, or adding sections to a survey can be doneeasily by a remove administrator for all users. The administrator simplyedits the form and it is automatically enabled for anyone already usingthe administrator's module through the application 232.

FIG. 4 is a diagram that illustrates an example of a configurationinterface 410 for customizing precision levels for data to be collectedon an electronic form. The configuration interface 410 enables theadministrator to configure different precision levels of informationshown on a customized electronic form. As described previously withrespect to FIG. 1, such configurations can be used to cause thecustomized electronic form to dynamically adjust to one of the specifiedprecision levels as the user is interacting with the electronic form.

In the example depicted, the configuration interface 410 allows theadministrator to specify the data that is requested from the user (e.g.,weight, pain, diet information). In region 410, the administrator canspecify the level of precision for the information requested on theelectronic form. For example, the electronic form may request anincrease or decrease in weight, the actual number corresponding to theuser's weight, or an estimate of the user's weight to the nearestpounds.

In region 414, the administrator may specify a set of requirements thatcustomize the electronic form for a particular user or a particulargroup of users. For example, an administrator may specify a target levelfor the burden imposed on the user (e.g., the difficulty for the user inproviding the information requested on the electronic form). As anotherexample, the administrator may indicate a target level for a completionspeed (e.g., the time it takes for the user to complete the electronicform), and a requirement for how complete responses need to be in orderto be considered a satisfactory or valid user input.

The set of requirements provided on the configuration interface 410 maythen be used to automatically and dynamically configure and adjust theelectronic form on the application 232 based on data associated with theuser. Because the requirements are different for each user, theapplication 232 may variably adjust the electronic form based on thedata associated with each individual user. For example, an administratormay use the configuration interface 410 to create an electronic formwith a “low burden” requirement. When the electronic form is generatedon the application 232, the low burden requirement for a younger usermay be higher than the low burden requirement for an older user withdisabilities or potential memory loss. Thus, in this example, theapplication 232 may use demographic information associated with the userto variably construct a similarly configured electronic form to providedifferent user experiences.

In region 416, the administrator can configure specific rules thatcoordinate the configuration and/or adjustment of an electronic form inrelation to specified triggers and conditions associated with theelectronic form. For example, the configuration interface 410 mayprovide a drop-down list that allows the administrator to select from alist of triggers and a list of conditions. Examples of triggers includereceiving sensor data indicating that the user has recently fallen,receiving user input data indicating an abnormally high pain levelmeasurement, and/or data from a wearable device indicating insufficientphysical activity. Examples of conditions may include determining thatthe user's historical activity pattern is below an anticipated goalspecified by a treatment program, data indicating that the user hasfailed to sufficiently complete the last four electronic forms, amongothers.

The administrator can then specify actions to be performed by theapplication 232 in response to a satisfaction of the triggers and/or theconditions. The administrator can either specify application-specificactions (e.g., ending a program early), or actions to be performed inrelation to a particular electronic form (e.g., adding a new segment tothe electronic form, providing an alternative form, etc.). In thisregard, the associated rules of the electronic form can be used todynamically adjust the content provided on the electronic form and/orthe presentation of the electronic form in relation to presentconditions associated with the user based on recent data received on theapplication 232.

The application 232 may customize the content and appearance of a formfor a particular user based on, for example, a language or dialect ofthe user, a reading level of the user, a health literacy of the user,impairments of the user (color blindness, etc.), a technology fluency ofthe user, operating system conventions, a current context of the user orthe user device, and other factors. For example, the application 232 maystore, or access from the application server 220, a user profile for theuser to obtain scores for any or all of the factors indicated above. Theapplication 232 may use the scores for these user-specific factors toapply rules embedded in the received form definitions. A form definitionmay include a rule that selects among different text prompts to providebased on a language of the user and a reading level of the user, thusproviding an understandable instruction to each user despite the widelyvarying characteristics among the user base. The application 232 itselfmay apply general rules separate from form definitions, for example, toadjust form appearance according to user preferences in the application232, to adjust form presentation for the characteristics of a userdevice (e.g., screen size, resolution, etc.), to select content that isin the user's language, and so on. Additional changes that can be made,based on rules in form definitions or by the application 232, includepresenting text in the user's language, setting different font sizesdepending on a user's age, phrasing questions differently depending onthe user's gender, and phrasing questions differently depending on auser's age and/or reading level. A form definition may provide multipledifferent variations, such as multiple phrasings of a question, withrules indicating when each version is appropriate. The application 232applies those rules to select and present the content most appropriatefor the user each time a view of the form is displayed.

FIG. 5 is a diagram that illustrates an example of a process 500 forprocessing user input to an electronic form in relation to a set ofhierarchical data attributes. The process 500 is generally used tostandardize different types of user inputs provided on the customizedelectronic forms in order to enable the system to track patternsassociated with user activity, user behavior, and/or user patterns. Forinstance, different electronic forms can be configured to collectdifferent types of information. As an example, one form can be used toevaluate user activity level by collecting exercise data submitted bythe user, whereas another form can be used to track caloric intake bycollecting dietary information submitted by the user. The process 500can be used to standardize the data collected by different electronicforms, or different parts of a single electronic form so that the systemcan then make diagnostic inferences and/or determinations based on thecollected data across multiple forms.

In general, collected data is standardized categorizing different datatypes within hierarchies of information, and then obtaining attributesthat can be then be analyzed by the system in relation to attributes ofanother type of input. For example, a user's exercise journal caninclude text segments corresponding to different exercises performed bythe user. The system can then identify the average number of caloriesburned by performing each exercise in order to standardize the textualinput in a number of calories burned by the user as a result ofperforming the exercises. In another example, a user's dietary journalwith foods consumed can be standardized by identifying the number ofcalories consumed by the user. The system can finally then relate thedifferent types of user input provided on the exercise journal and onthe dietary journal by relating the calories consumed and the caloriesburned.

In the example depicted in FIG. 5, the electronic form 510 is initiallypresented to a user to request information relating to his diet. Theuser provides a text query of “1 APPLE” into a text field to indicatethat he has eaten an apple. The application 232 then processes thereceived text in relation to a hierarchy of categories 520. Forinstance, the application 232 may use natural language processingtechniques to identify the query “APPLE” as corresponding to an element522 within the hierarchy 520.

Each element within the hierarchy 520 includes a set of attributes thatare generated based on the categories represented within the hierarchy.In the example, the hierarchy categorizes different of food groups, suchas meats and fruit, and further categorizes each food groups intodifferent constituent that fit within the same category (e.g., differenttypes of fruit). Each element within the hierarchy 520 is associatedwith a set of nutritional attributes that relate to, for example, theestimated number of calories, total fat, sedum, total carbohydrates, andprotein are included in one serving of each element.

In response to determining that the text submitted by the usercorresponds to the element 522, the application 232 then obtains theinformation included within the set of attributes 530. The applicationthen generates a table 542 that includes information related to the userinput (e.g., user and date), the input provided by the user, and the setof attributes associated with the corresponding element within thehierarchy 520. The table 542 is then stored in user data 540, whichstores user input data for analyzing user input on the electronic form510.

The conversion to a common data format, or the assignment of a categoryor data type to an input, can be done by the user device and/or by theserver. In some implementations, there are multiple databases availablefor administrators to select data types and category hierarchies from,e.g., a food database, a medication database, a fact database, and soon.

In some implementations, individual data types can be defined, and eachdata type can have a number of data type properties defined. The datatype properties may represent an amount or other value obtained throughuser input, sensor data, or other sources. The data type properties canbe assigned values based on the acquired information, e.g., to representa number food servings, a number of caplets for medication, a bodyweight, and so on. A data type can have other properties defined, forexample, pre-programmed information about a typical calorie content,sugar content, etc., of a food can be stored in association with thatfood's data type, to provide information for all instances of the datatype. A data type may have a conversion parameter, a unit specification(e.g., milligrams, meters, calories, etc.), and/or other informationassociated with it. Other information may be associated with a datatype, such as an identifier for obtaining external information, e.g., arecord in another database, a URL for an image file representing theitem corresponding to the data type, and so on. Groups of one or moredata types can be organized into data type categories (e.g., fruit, painmedication, body vitals, etc.), and groups of one or more data typecategories can be organized into data type root categories (e.g., food,medication, measurement, etc.). These hierarchies can provideconsistency across the data gathered from different versions ofelectronic forms and across electronic forms used by differentorganizations.

Each entry can be associated with one or more higher-level data typecategories, such as food, medication, body vital information, and so on.

The processing technique depicted in FIG. 5 illustrates how theapplication 232 can obtain information related to a user input providedon an electronic form, identify how the related information iscategorized within a corresponding hierarchy, and store the informationusing relationships of categories. In this regard, user inputs ofdifferent data formats can be converted to a common data format based onusing different hierarchies coincide with specific data formats. Forexample, input relating to a user's diet, as illustrated in FIG. 4, canbe converted based on nutritional attributes of food included in theuser's diet, whereas input relating to a user's activity level can beconverted based on the calorie loss associated with exercises performedby the user. The converted information can then be compared and/orevaluated in order to make diagnostic inferences and/or determinationsrelating to the user's progress in relation a particular healthcondition.

As another example, an administrator that creates an electronic form candefine a data type of “weight.” Weight data type can be collected frommany different electronic forms, in a variety of formats. In one form, auser may be prompted to manually enter a weight value via a keypad. Inanother form, a user may be prompted to manually select their weightfrom a scrollable wheel. In both cases, the form definition will specifythat the collected value should be stored as a “weight” data type. Theclient device and/or the server system it communicate with can alsoconvert varied inputs (e.g., text, button selection, keypad entries,etc.) to a consistent standard format. In this manner, the user-facingexperiences can be customized for the audience, while retaining a clean,concise data structure. In addition, by abstracting the user-facinglayer of the experience from the underlying data structure, the systemcan provide an essentially unlimited personalization capability whileretaining the ability to perform disciplined data science at the server.

FIG. 6 is a diagram that illustrates an example of a performanceanalysis report 610 for an electronic form. The report 610 provides theadministrator with information relating to how a user or a collection ofusers performed on a particular electronic form (e.g., referred to as“Form A”). In the examples depicted, the report 610 includes a region612 representing overall completion rate, a region 614 representing anaverage time spent completing the form, and a region 616 representingsuggested form updates across all users that completed the form.

The report 610 includes additional performance information for users oftwo age demographics (e.g., younger than 50 years, and older than 50years old) and compares the performance with respect to the performancecategories provided. As illustrated, users that were older than 50 yearshad greater difficulty completing Form A compared to users that wereyounger than 50 years (e.g., 62% compared to 96%). In addition, usersthat were older than 50 years old spent 8 more minutes on Form A onaverage compared to users that were younger than 50 years old (e.g., 11minutes compared to 3 minutes).

As described above with respect to FIG. 2, the performance data relatingto the electronic forms can be used to provide automated recommendationswithin a performance report. In the example, because users that wereolder than 50 years had a lower rate of completion and a higher timespent on Form A, the system determines that the current configuration ofthe form may be too complex for users within this age demographic. Thus,the report 610 includes a recommendation to reduce the form complexityin order to improve performance of users within this age demographic onthe form. In comparison, due to high completion rate and low amount oftime spent on the form by users that were younger than 50 years, thereport 610 includes a recommendation to maintain the current form forusers within this demographic.

Performance reports such as the report 610 can be generated by theapplication server 220 or the administrator system 210 in order toprovide administrators with feedback on customized electronic forms thatare designed on the configuration interface 212 and displayed to usersthrough the application 232. In this regard, information included withinthe performance reports can be used to adjust the form definitions ofthe customized electronic forms in order to improve user engagement onthe application 232. As described above, in some instances, theperformance information can be used to allow manual adjustment by theadministrators through the configuration interface 212, whereas in otherinstances, the performance information can be used in automatedalgorithms to dynamically update the form definitions without anyintervention by the administrators. In the second instance, periodicautomatic adjustments to a customized electronic form can be used tofine-tune or tailor the electronic form specifically towards the needsof an individual user or a collection of users based on prior user inputdata received on prior instances of the electronic form.

In general, the performance reports can indicate measures (e.g., scores,charts, graphics, etc.) that indicate the feasibility, validity,usefulness, and usability of forms that an administrator has designed.

FIG. 7A is a diagram that illustrates an example of a process 700A fordisplaying a customized electronic form to a user. Briefly, the process700A may include receiving form data that specifies characteristics ofan electronic form (710), obtaining data indicating characteristics oractivities of a user (720), selecting an interactive element based onthe form data and the user data (730), customizing a view of theelectronic form for the user (740), and displaying the customized viewof the electronic form (750).

In more detail, the process 700A may include receiving form data thatspecifies characteristics of an electronic form (710). For instance, theuser device 230 may receive form data that specifies characteristics ofan electronic form over the network 205. As described above, the formdata can include rules configured to vary the content of the electronicform that is presented for different users. The characteristics of theform can include a desired level of precision for information presentedon the electronic form, the way in which input data is received by theelectronic form, actions to be performed in response to a satisfactionof triggers and/or conditions specified for the rules, the sequence ofsections within the electronic form, and/or the content that is providedon the electronic form. In some instances, the form data may also definevalidation rules that specify requirements for user input on theelectronic form.

In some implementations, the form data is configured to cause the userdevice 230 to obtain user input to the electronic form, and additionaldata from additional sources (e.g., passively-sensed sensor data,actively-sensed sensor data, data from a sensor of the user device 230,data from a device wirelessly connected to the user device 230, or datafrom a third-party API). As described above the form data can be used todirect the user device 230 to obtain such data in response to certainconditions and/or triggers specified by rules included within the formdata.

The form data includes various types of rules as described above. In oneexample, a rule can specify one or more triggers to initiate systemactions specified by the rule, one or more conditions to evaluate forthe rule, and one or more actions to be performed based on thecondition. In some instances, the rules can include a validation rulethat specifies requirements for user input to a portion of theelectronic form. As described above, the validation rules can be used todetermine that a user input does not satisfy a set of requirementsindicated by the validation rule, and in response, the application 232may restrict submission of data through the electronic form based ondetermining that the user input does not satisfy the validation rule. Asan example, if an electronic form requests a user to complete anassessment, and the user provides incorrect and/or insufficientresponses, the application 232 may automatically restrict further inputfrom the user based on determining, based on the failure of the userinput to satisfy a validation rule, that the user is probably confusedby the way in which the electronic form is displayed on the application232.

In some implementations, the form data specifies a data type for userinput collected through the electronic form. The form data can then beused to select an interactive element to receive user input of thespecified data type. In such implementations, the interactive elementcan be selected from among different interactive elements that eachreceive input of the specified data type. The selected interactiveelement can then be included within a customized view of the electronicform. As an example, the form data can indicate the collection ofnumerical data for a weight assessment form. The application 232 canthen select from among different types of data entry fields that eachcollect numerical input from the user but with different levels ofprecision as illustrated in FIG. 1. In this example, the application 232may select a data entry field that represents the appropriate level ofprecision based on variety of factors such as, for example, demographicinformation of the user or previous inputs from the user indicating alevel of comprehension relating to the electronic form.

In some implementations, the electronic form includes multiple sectionsdisplayed in a sequence. The content provided in the sections can becustomized based on the rules that included within the form definitionfor the electronic form. For example, the rules may specify the displayof types of content on a particular section in response to receivinguser input on the application 232 in relation to the particular section.

The process 700A may include obtaining data indicating characteristicsor activities of a user (720). For instance, the user device 230 mayreceive user data indicating characteristics or activities of the user.As described above, user characteristics can include demographicinformation associated with the user, a medical history/conditionassociated with the user, and/or user preferences for engagement throughthe application 232. Examples of user activities can include input datareceived from devices wirelessly connected to the user device 230 (e.g.,pedometer data, sleep pattern data), user inputs provided on theapplication 232 (e.g., prior text queries, submitted search queries,prior user selections), data received from third party APIs (e.g., listof background apps used by the user while completed the electronic form,data associated with a social media profile), or data obtained fromsensors of the user device 230 (e.g., accelerometer data, location data,microphone data).

The process 700A may include selecting an interactive element based onthe form data and the user data (730). For instance, the application 232may select an interactive element from among a set of multipleinteractive elements based on the user data and the rules in thereceived from data. For example, this may refer to selecting aparticular interaction element from a set of alternative interactionelements based on the user's preferences for engagement on theapplication 232 (e.g., selecting a slider interface over a text inputbox based on a user data indicating greater frequency of usage of theformer over the later. In another example, this may refer to selectingan optimal precision level for information to be provided on theelectronic form based on requirements set forth by the administrator(e.g., maximal user burden, requested completion rate, speed ofcompletion). In this example, the application 232 may dynamicallyidentify an optimal level of precision based on the informationindicated by the user data, and select from among alternative formconfigurations of different precision levels.

The process 700A may include customizing a view of the electronic formfor the user (740). For instance, the user device 230 may customize aview of the electronic form for the user by including, in the customizedview, the interactive element selected based on the user data and therules in the received form data. For example, the selected interactiveelement may be used to dynamically construct or adjust the display theelectronic form on the application 232. As described above, in someinstances, this process may be performed in real-time such that thecustomization of the electronic form may be performed while the user ispresently using the application 232 without requiring an additionalapplication-wide update from the application server 220.

The process 700A may include displaying the customized view of theelectronic form (750). For instance, the user device 230 may display theview of the customized electronic form on the application 232.

FIG. 7B is a diagram that illustrates an example of a process 700B forgenerating a form definition for customizing an electronic form.Briefly, the process 700B may include providing an interface fordesigning an electronic form (712), receiving data indicating user inputthrough the user interface (722), generating form data that specifiesrules for customizing the electronic form (732), and providing the formdata to an electronic computing device (742).

In more detail, the process 700B may include providing an interface fordesigning an electronic form (712). For instance, the administratorsystem 210 may provide a configuration interface 212 for designing anelectronic form for collecting data about the user of the user device230. As described previously with respect to FIGS. 3 and 4, theconfiguration interface 212 enables the administrator to customize aform definition that configures or adjusts the display of the electronicform on the application 232.

The process 700B may include receiving data indicating user inputthrough the user interface (722). For instance, the administrator system210 may receive data indicating user input by the administrator throughthe configuration interface. The user input may indicate data types fordata to collect using the electronic form and layout parameters for theelectronic form. As described above, the administrator may specify datatypes for user input (e.g., numerical input, textual input, voice input)based on the purpose of the electronic form. The administrator may alsospecify layout parameters that adjust the presentation of the electronicform and/or the interactivity of the electronic form with the userthrough the application 232. For example, the administrator may selectinteractive elements to include within the electronic form, adjust theadjustment of the selected interactive elements in a particular layout,and/or configure other graphical attributes associated with theelectronic form.

The process 700B may include generating form data that specifies rulesfor customizing the electronic form (732). For instance, theadministrator system 210 may generate form data that specifies rules forcustomizing the electronic form. As described above, the form dataincludes a form definition that includes, among other types ofspecifications, rules that adjust the configuration of the electronicform based on the satisfaction of associated triggers and/or conditionsthat are specified by the administrator. As an example, a rule can beused to change the content provided on the electronic form based on thesatisfaction of triggers and/or conditions that are each associated withdifferent types of users. In this regard, the rules specified by theform definition can be used to vary content provided on the sameelectronic form that are provided to different users.

The process 700B may include providing the form data to an electroniccomputing device (742). For instance, the administrator system 210 orthe application server 220 may transmit the form data to the user device230 to configure or adjust the display of the electronic form on theapplication 232. As described above with respect to FIG. 2, the formdata that is provided can be used to initially configure a newelectronic form to be displayed within the application 232, or adjustthe configuration of an existing electronic form that has beenpreviously displayed to the user. In the second instance, thetransmitted form data can be processed by the user device in real-timein order to dynamically adjust the existing electronic form in real-timewithout any intervention by the user. This technique can be used toautomatically adjust the electronic form in response to user input datareceived on the application 232, or manually adjust the electronic formsin response to updates to the form definition provide by theadministrator on the configuration interface 212.

FIG. 8 is a block diagram of computing devices 800, 850 that can be usedto implement the systems and methods described in this document, aseither a client or as a server or plurality of servers. Computing device800 is intended to represent various forms of digital computers, such aslaptops, desktops, workstations, personal digital assistants, servers,blade servers, mainframes, and other appropriate computers. Computingdevice 850 is intended to represent various forms of mobile devices,such as personal digital assistants, cellular telephones, smartphones,and other similar computing devices. Additionally, computing device 800or 850 can include Universal Serial Bus (USB) flash drives. The USBflash drives can store operating systems and other applications. The USBflash drives can include input/output components, such as a wirelesstransmitter or USB connector that can be inserted into a USB port ofanother computing device. The components shown here, their connectionsand relationships, and their functions, are meant to be exemplary only,and are not meant to limit implementations of the inventions describedand/or claimed in this document.

Computing device 800 includes a processor 802, memory 804, a storagedevice 806, a high-speed interface 808 connecting to memory 804 andhigh-speed expansion ports 810, and a low speed interface 812 connectingto low speed bus 814 and storage device 806. Each of the components 802,804, 806, 808, 810, and 812, are interconnected using various busses,and can be mounted on a common motherboard or in other manners asappropriate. The processor 802 can process instructions for executionwithin the computing device 800, including instructions stored in thememory 804 or on the storage device 806 to display graphical informationfor a GUI on an external input/output device, such as display 816coupled to high speed interface 808. In other implementations, multipleprocessors and/or multiple buses can be used, as appropriate, along withmultiple memories and types of memory. Also, multiple computing devices800 can be connected, with each device providing portions of thenecessary operations, e.g., as a server bank, a group of blade servers,or a multi-processor system.

The memory 804 stores information within the computing device 800. Inone implementation, the memory 804 is a volatile memory unit or units.In another implementation, the memory 804 is a non-volatile memory unitor units. The memory 804 can also be another form of computer-readablemedium, such as a magnetic or optical disk.

The storage device 806 is capable of providing mass storage for thecomputing device 800. In one implementation, the storage device 806 canbe or contain a computer-readable medium, such as a floppy disk device,a hard disk device, an optical disk device, or a tape device, a flashmemory or other similar solid state memory device, or an array ofdevices, including devices in a storage area network or otherconfigurations. A computer program product can be tangibly embodied inan information carrier. The computer program product can also containinstructions that, when executed, perform one or more methods, such asthose described above. The information carrier is a computer- ormachine-readable medium, such as the memory 804, the storage device 806,or memory on processor 802.

The high speed controller 808 manages bandwidth-intensive operations forthe computing device 800, while the low speed controller 812 manageslower bandwidth intensive operations. Such allocation of functions isexemplary only. In one implementation, the high-speed controller 808 iscoupled to memory 804, display 816, e.g., through a graphics processoror accelerator, and to high-speed expansion ports 810, which can acceptvarious expansion cards (not shown). In the implementation, low-speedcontroller 812 is coupled to storage device 806 and low-speed expansionport 814. The low-speed expansion port, which can include variouscommunication ports, e.g., USB, Bluetooth, Ethernet, wireless Ethernetcan be coupled to one or more input/output devices, such as a keyboard,a pointing device, microphone/speaker pair, a scanner, or a networkingdevice such as a switch or router, e.g., through a network adapter. Thecomputing device 800 can be implemented in a number of different forms,as shown in the figure. For example, it can be implemented as a standardserver 820, or multiple times in a group of such servers. It can also beimplemented as part of a rack server system 824. In addition, it can beimplemented in a personal computer such as a laptop computer 822.Alternatively, components from computing device 800 can be combined withother components in a mobile device (not shown), such as device 850.Each of such devices can contain one or more of computing device 800,850, and an entire system can be made up of multiple computing devices800, 850 communicating with each other.

The computing device 800 can be implemented in a number of differentforms, as shown in the figure. For example, it can be implemented as astandard server 820, or multiple times in a group of such servers. Itcan also be implemented as part of a rack server system 824. Inaddition, it can be implemented in a personal computer such as a laptopcomputer 822. Alternatively, components from computing device 800 can becombined with other components in a mobile device (not shown), such asdevice 850. Each of such devices can contain one or more of computingdevice 800, 850, and an entire system can be made up of multiplecomputing devices 800, 850 communicating with each other.

Computing device 850 includes a processor 852, memory 884, and aninput/output device such as a display 854, a communication interface866, and a transceiver 888, among other components. The device 850 canalso be provided with a storage device, such as a microdrive or otherdevice, to provide additional storage. Each of the components 850, 852,884, 854, 866, and 888, are interconnected using various buses, andseveral of the components can be mounted on a common motherboard or inother manners as appropriate.

The processor 852 can execute instructions within the computing device850, including instructions stored in the memory 884. The processor canbe implemented as a chipset of chips that include separate and multipleanalog and digital processors. Additionally, the processor can beimplemented using any of a number of architectures. For example, theprocessor 810 can be a CISC (Complex Instruction Set Computers)processor, a RISC (Reduced Instruction Set Computer) processor, or aMISC (Minimal Instruction Set Computer) processor. The processor canprovide, for example, for coordination of the other components of thedevice 850, such as control of user interfaces, applications run bydevice 850, and wireless communication by device 850.

Processor 852 can communicate with a user through control interface 858and display interface 856 coupled to a display 854. The display 854 canbe, for example, a TFT (Thin-Film-Transistor Liquid Crystal Display)display or an OLED (Organic Light Emitting Diode) display, or otherappropriate display technology. The display interface 856 can includeappropriate circuitry for driving the display 854 to present graphicaland other information to a user. The control interface 858 can receivecommands from a user and convert them for submission to the processor852. In addition, an external interface 882 can be provide incommunication with processor 852, so as to enable near areacommunication of device 850 with other devices. External interface 882can provide, for example, for wired communication in someimplementations, or for wireless communication in other implementations,and multiple interfaces can also be used.

The memory 884 stores information within the computing device 850. Thememory 884 can be implemented as one or more of a computer-readablemedium or media, a volatile memory unit or units, or a non-volatilememory unit or units. Expansion memory 884 can also be provided andconnected to device 850 through expansion interface 882, which caninclude, for example, a SIMM (Single In Line Memory Module) cardinterface. Such expansion memory 884 can provide extra storage space fordevice 850, or can also store applications or other information fordevice 850. Specifically, expansion memory 884 can include instructionsto carry out or supplement the processes described above, and caninclude secure information also. Thus, for example, expansion memory 884can be provide as a security module for device 850, and can beprogrammed with instructions that permit secure use of device 850. Inaddition, secure applications can be provided via the SIMM cards, alongwith additional information, such as placing identifying information onthe SIMM card in a non-hackable manner.

The memory can include, for example, flash memory and/or NVRAM memory,as discussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 884, expansionmemory 884, or memory on processor 852 that can be received, forexample, over transceiver 888 or external interface 882.

Device 850 can communicate wirelessly through communication interface866, which can include digital signal processing circuitry wherenecessary. Communication interface 866 can provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others.Such communication can occur, for example, through radio-frequencytransceiver 888. In addition, short-range communication can occur, suchas using a Bluetooth, Wi-Fi, or other such transceiver (not shown). Inaddition, GPS (Global Positioning System) receiver module 880 canprovide additional navigation- and location-related wireless data todevice 850, which can be used as appropriate by applications running ondevice 850.

Device 850 can also communicate audibly using audio codec 880, which canreceive spoken information from a user and convert it to usable digitalinformation. Audio codec 880 can likewise generate audible sound for auser, such as through a speaker, e.g., in a handset of device 850. Suchsound can include sound from voice telephone calls, can include recordedsound, e.g., voice messages, music files, etc. and can also includesound generated by applications operating on device 850.

The computing device 850 can be implemented in a number of differentforms, as shown in the figure. For example, it can be implemented as acellular telephone 480. It can also be implemented as part of asmartphone 882, personal digital assistant, or other similar mobiledevice.

Various implementations of the systems and methods described here can berealized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations of suchimplementations. These various implementations can includeimplementation in one or more computer programs that are executableand/or interpretable on a programmable system including at least oneprogrammable processor, which can be special or general purpose, coupledto receive data and instructions from, and to transmit data andinstructions to, a storage system, at least one input device, and atleast one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium”“computer-readable medium” refers to any computer program product,apparatus and/or device, e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs), used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device,e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitorfor displaying information to the user and a keyboard and a pointingdevice, e.g., a mouse or a trackball by which the user can provide inputto the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback, e.g., visual feedback,auditory feedback, or tactile feedback; and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component, e.g., as a dataserver, or that includes a middleware component, e.g., an applicationserver, or that includes a front end component, e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here, or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication, e.g., acommunication network. Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

A number of embodiments have been described. Nevertheless, it will beunderstood that various modifications can be made without departing fromthe spirit and scope of the invention. In addition, the logic flowsdepicted in the figures do not require the particular order shown, orsequential order, to achieve desirable results. In addition, other stepscan be provided, or steps can be eliminated, from the described flows,and other components can be added to, or removed from, the describedsystems. Accordingly, other embodiments are within the scope of thefollowing claims.

What is claimed is:
 1. A method performed by one or more computers, themethod comprising: providing, by the one or more computers, survey dataspecifying content of a survey to each of multiple devices that arerespectively associated with different users; receiving, by the one ormore computers, response data from the devices over a computer network,the response data indicating (i) survey responses of the users and (ii)amounts of time spent by the users to interact with or reach completionof the survey presented based on the survey data; evaluating, by the oneor more computers, the response data to determine measures of amounts oftime spent on the survey by different groups of users, wherein thegroups are based on user characteristics of the users such that, in eachof the different groups, users in the group share one or more usercharacteristics associated with the group, and wherein the differentgroups are associated with different user characteristics; providing, bythe one or more computers, results of the evaluation that indicate thedetermined measures of amounts of time spent on the survey by thedifferent groups of users to a device of an administrator forpresentation on a user interface of the device; after providing theresults of the evaluation on the user interface, receiving, by the oneor more computers, data indicating user input through the user interfacethat indicates one or more changes to the survey; based at least in parton the user input, generating, by the one or more computers, a datapackage that includes (i) data specifying multiple configurations of thesurvey, and (ii) one or more rules for selecting among the multipleconfigurations of the survey based on the user characteristicsassociated with the respective groups of users, wherein the data packageis configured to cause devices receiving the data package to presentdifferent configurations of the survey to the different groups of users;and distributing, by the one or more computers, the generated datapackage to devices of users in the different groups over the computernetwork, the data package enabling each of the devices to (i) use theone or more rules in the data package to select, based on one or moreuser characteristics of a user of the device, from among the multipleconfigurations of the survey specified by the data package and (ii)present the configuration of the survey that the device selected basedon the one or more user characteristics of the user of the device. 2.The method of claim 1, comprising providing, to the device of theadministrator, data indicating a level of completion of one or moresurveys by the users.
 3. The method of claim 1, comprising providing, tothe device of the administrator, data indicating a survey completionrate for the users.
 4. The method of claim 1, wherein the results of theevaluation include data indicating an average amount of time spent byusers presentations of the survey based on the survey data.
 5. Themethod of claim 1, wherein evaluating the response data comprisesdetermining measures for survey interaction for different groups of theusers; and wherein providing the results of the evaluation comprisesproviding data indicating the different measures of survey interactionfor the different groups.
 6. The method of claim 1, wherein the surveydata is configured to cause devices to customize presentation of thesurvey so that at least some of the different users receive differentsurvey content during presentation of the survey.
 7. The method of claim1, wherein the survey data is configured to provide one or more surveysfor health monitoring, medical treatment, or health research.
 8. Themethod of claim 1, wherein the survey data is configured to acquireinformation indicating at least one of physiological attributes ordisease symptoms of the users.
 9. The method of claim 1, wherein theresponse data includes one or more sensor measurements captured by thedevices of the users.
 10. The method of claim 1, wherein providing theresults of the evaluation comprises providing a completion rate for thesurvey for each of the different groups of users.
 11. A systemcomprising: one or more computers; and one or more computer-readablemedia storing instructions that, when executed by one or more computers,are operable to cause the one or more computers to perform operationscomprising: providing, by the one or more computers, survey dataspecifying content of a survey to each of multiple devices that arerespectively associated with different users; receiving, by the one ormore computers, response data from the devices over a computer network,the response data indicating (i) survey responses of the users and (ii)amounts of time spent by the users to interact with or reach completionof the survey presented based on the survey data; evaluating, by the oneor more computers, the response data to determine measures of amounts oftime spent on the survey by different groups of users, wherein thegroups are based on user characteristics of the users such that, in eachof the different groups, users in the group share one or more usercharacteristics associated with the group, and wherein the differentgroups are associated with different user characteristics; providing, bythe one or more computers, results of the evaluation that indicate thedetermined measures of amounts of time spent on the survey by thedifferent groups of users to a device of an administrator forpresentation on a user interface of the device; after providing theresults of the evaluation on the user interface, receiving, by the oneor more computers, data indicating user input through the user interfacethat indicates one or more changes to the survey; based at least in parton the user input, generating, by the one or more computers, a datapackage that includes (i) data specifying multiple configurations of thesurvey, and (ii) one or more rules for selecting among the multipleconfigurations of the survey based on the user characteristicsassociated with the respective groups of users, wherein the data packageis configured to cause devices receiving the data package to presentdifferent configurations of the survey to the different groups of users;and distributing, by the one or more computers, the generated datapackage to devices of users in the different groups over the computernetwork, the data package enabling each of the devices to (i) use theone or more rules in the data package to select, based on one or moreuser characteristics of a user of the device, from among the multipleconfigurations of the survey specified by the data package and (ii)present the configuration of the survey that the device selected basedon the one or more user characteristics of the user of the device. 12.The system of claim 11, wherein the operations comprise providing, tothe device of the administrator, data indicating a level of completionof one or more surveys by the users.
 13. The system of claim 11, whereinthe results of the evaluation include data indicating an average amountof time spent by users on surveys presented based on the survey data.14. The system of claim 11, wherein providing the results of theevaluation comprises providing a completion rate for the survey for eachof the different groups of users.
 15. One or more non-transitorycomputer-readable media storing instructions that, when executed by oneor more computers, are operable to cause the one or more computers toperform operations comprising: providing, by the one or more computers,survey data specifying content of a survey to each of multiple devicesthat are respectively associated with different users; receiving, by theone or more computers, response data from the devices over a computernetwork, the response data indicating (i) survey responses of the usersand (ii) amounts of time spent by the users to interact with or reachcompletion of the survey presented based on the survey data; evaluating,by the one or more computers, the response data to determine measures ofamounts of time spent on the survey by different groups of users,wherein the groups are based on user characteristics of the users suchthat, in each of the different groups, users in the group share one ormore user characteristics associated with the group, and wherein thedifferent groups are associated with different user characteristics;providing, by the one or more computers, results of the evaluation thatindicate the determined measures of amounts of time spent on the surveyby the different groups of users to a device of an administrator forpresentation on a user interface of the device; after providing theresults of the evaluation on the user interface, receiving, by the oneor more computers, data indicating user input through the user interfacethat indicates one or more changes to the survey; based at least in parton the user input, generating, by the one or more computers, a datapackage that includes (i) data specifying multiple configurations of thesurvey, and (ii) one or more rules for selecting among the multipleconfigurations of the survey based on the user characteristicsassociated with the respective groups of users, wherein the data packageis configured to cause devices receiving the data package to presentdifferent configurations of the survey to the different groups of users;and distributing, by the one or more computers, the generated datapackage to devices of users in the different groups over the computernetwork, the data package enabling each of the devices to (i) use theone or more rules in the data package to select, based on one or moreuser characteristics of a user of the device, from among the multipleconfigurations of the survey specified by the data package and (ii)present the configuration of the survey that the device selected basedon the one or more user characteristics of the user of the device. 16.The one or more non-transitory computer-readable media of claim 15,comprising providing, to the device of the administrator, dataindicating a level of completion of one or more surveys by the users.17. The one or more non-transitory computer-readable media of claim 15,wherein the results of the evaluation include data indicating an averageamount of time spent by users on surveys presented based on the surveydata.
 18. The one or more non-transitory computer-readable media ofclaim 15, wherein providing the results of the evaluation comprisesproviding a completion rate for the survey for each of the differentgroups of users.